Video tracking with EthoVision XT
EthoVision XT is an automated video tracking and motion analysis system. Video tracking means that EthoVision XT detects and follows one or more animals, or objects, in a video image (live footage or pre-recorded video file), and tracks their location and movement.
EthoVision XT is a software platform. The Base version help you carry out tests like the Open field and the Morris water maze test. With its Add-on modules, it offers a wide range of video tracking options, and extensive analysis of locomotion and behavior. For more information, see Modules of EthoVision XT .
The entire process carried out by EthoVision may be summarized as follows: a video camera observes the movement of one or more subjects, and passes images of the subjects to your computer. There they are transformed into a digital signal and optionally encoded as a digital video file. From this digital signal or file, EthoVision first detects the subjects and then extracts the size of the subjects and position of one or more body points of the subjects in each image. That data is then transformed into a series of dependent variables quantifying the behavior of the subjects.
See also
▪The EthoVision XT 19 - Quick Start Guide, which came with the software.
▪The EthoVision XT Video Tutorial to learn how to set up an experiment in EthoVision XT. In EthoVision XT, choose Help > Video Tutorial. Note that the Video Tutorial contains audio.
Image sensing
The starting point of an imaging system is a video camera. A video camera transforms a scene (the area in front of the lens), into an image (a picture taken by the camera). If you have an analog camera the image must be converted into a digital image consisting of pixels. With USB, Gigabit Ethernet (GigE) and Internet Protocol (IP) cameras, digitization occurs within the camera. You can plug them directly into your computer. Instead of using live video from a camera, you can also make a digital video file and use that for tracking.
One video image from a camera or digital video file is called a frame. The frame is made by a point which scans the scene in a series of horizontal lines (called fields), starting at the top and working its way down to the bottom. The fields are interlaced, that is, the camera first scans the odd lines, then the even lines. As the scene is scanned, the brightness (or color) of the scene is transformed into an analog signal describing the intensity of the image at each point of the scan.
The number of frames scanned in each second is called the frame rate, and this determines the maximum possible sample rate for EthoVision. The frame rate differs between cameras. See Cameras supported by EthoVision XT
When EthoVision XT receives the video frame, it does not “see” a mouse in an open field (for instance), but a bitmap composed of pixels, each of which has a particular gray value. The first thing that EthoVision XT does is to distinguish between the subjects to be tracked, and the background. In order to establish which pixel is part of the animal and which pixel is part the background, EthoVision makes use of different detection methods. You can choose which detection method to use, but you can also let EthoVision XT select the best method for you. To do so, choose the Automated Setup of the Detection Settings.
Excluding noise
Whichever detection method you choose, it is possible that some pixels are identified as the subject, that are in fact just system noise or reflections. You can exclude these as follows:
▪Before tracking:
In the Arena Settings, define an arena. This means that pixels outside the arena are ignored.
In the Detection Settings, make various settings to exclude noise.
▪After tracking:
Smooth the track to filter out system noise, outliers and small movements.
EthoVision XT can track more than one animal per arena. A few techniques are available, from the basic tracking of unmarked subjects to the state-or-the-art identification technique based on Deep learning (see below). To track multiple subjects, you need the Social interaction add-on module. Modules of EthoVision XT
Tracking of unmarked subjects
You can track multiple animals without using color markers. For instance, track a shoal of fish to measure the average between-individual distance. However, keep in mind that there may be subject identity swaps, that is, what is labeled subject 1 may be labeled subject 2 later in the track. Therefore individual identification is not guaranteed. The methods listed below keep track of the subjects’ identities.
Color-marker tracking
Here below, you see an example of color marker tracking. EthoVision XT follows the color spots on the back of the insects, ignoring the shape of the animals.
Marker-assisted identification
With Marker-assisted identification, EthoVision XT uses the colors to identify the subjects, but tracks their entire body. This method allows to collect detailed data in social interaction tests, particularly in rodents. Just like for the previous method, you need to tell EthoVision XT which colors to follow.
Tracking based on Deep learning
If two subjects look different, or are marked in some way (also in grayscale video), EthoVision XT can discriminate between the two using its neural network. In most cased you need to mark one of the two subjects on its back or at the base of its tail.
Individual discrimination is accomplished after tracking, in a process named Data Preparation which involves a thorough review of the tracks and the corresponding video images. The neural network re-assigns the identity labels to the subjects based on their visual appearance. Note that to use Deep learning you need a compatible secondary graphics card (GPU). See Deep learning: Basics and Deep learning: Requirements
Video tracking and video recording
You can set EthoVision XT to track the subjects and, at the same time, save the video image to a digital video file. Alternatively, you can choose to save video first, and track the subjects later from that video. See Important things to know about data acquisition.
Subject position, size and orientation
When EthoVision XT has identified a group of adjacent pixels as a subject, it extracts a few features (depending on which add-on you have):
▪Subject position. This is the x,y coordinates of the body points being tracked: either the center-point only, that is the point mathematically in the center of the shape considered to be the subject, or the center-point, the nose-point and the tail-base point. See Overview of nose-tail base detection
▪Subject area. This is the area of the surface in the video image (camera image or video file) which composes the detected subject. The area is expressed and exported as squared distance units.
▪Subject area change. This is the proportion of the subject’s area that changes between frames. It is used to estimate the subject’s mobility.
▪Subject’s head orientation. When you track the subject’s nose point (only for contour-based methods), the software estimates the direction of the gaze. This is indicated with the blue line that originates from the nose point.
▪Subject elongation. This quantifies how stretched the subject’s body is.
For an overview of the output variables related to the subject’s body, see Dependent Variables in Detail > Body
The output of video tracking is a series of numbers representing the x,y coordinates and size of the subject. From the raw data EthoVision XT calculates a number of Dependent variables describing the behavior of the subject. These include speed and distance moved but also complex variables such as the direction of movement relative to an object.
Furthermore, with the Rat and Mouse Behavior Recognition modules, behaviors like grooming and rearing can be detected automatically.
Data analysis and results export
For all the dependent variables, EthoVision XT can calculate a wealth of descriptive statistics (such as mean, standard deviation, etc.). These are shown for each trial, and for the groups of trials that you may want to define (for example, treated vs. vehicle subjects).
Results and raw data (x,y coordinates and per-sample values of dependent variables) can be exported to text and Excel format.
note With custom code written in JavaScript language within EthoVision XT, you can greatly expand analysis capabilities. See a few examples in the following topics: